Neural Learning and Weight Flow on Stiefel Manifold
نویسندگان
چکیده
The aim of this paper is to present a new class of learning models for linear as well as non-linear neural layers called Orthonormal StronglyConstrained (SOC or Stiefel). They allow to solve orthonormal problems where orthonormal matrices are involved. After general properties of the learning rules belonging to this new class are shown, examples derived independently or by reviewing learning theories known from the literature are presented and discussed.
منابع مشابه
A Theory for Learning by Weight Flow on Stiefel-Grassman Manifold
Recently we introduced the concept of neural networks learning on Stiefel-Grassman manifold for MLP-like networks. Contributions of other authors have also appeared in the scientiic literature about this topic. The aim of this paper is to present a general theory for it, and to illustrate how existing theories may be explained within the general framework proposed here.
متن کاملA Neural Stiefel Learning based on Geodesics Revisited
In this paper we present an unsupervised learning algorithm of neural networks with p inputs and m outputs whose weight vectors have orthonormal constraints. In this setting the learning algorithm can be regarded as optimization posed on the Stiefel manifold, and we generalize the natural gradient method to this case based on geodesics. By exploiting its geometric property as a quotient space: ...
متن کاملSingular Value Decomposition Learning on Double Stiefel Manifold
The aim of this paper is to present a unifying view of four SVD-neural-computation techniques found in the scientific literature and to present some theoretical results on their behavior. The considered SVD neural algorithms are shown to arise as Riemannian-gradient flows on double Stiefel manifold and their geometric and dynamical properties are investigated with the help of differential geome...
متن کاملA Minor Subspace Algorithm Based on Neural Stiefel Dynamics
In the present paper we investigate iterative minor subspace analysis computation by describing a neural approach based on weight flow on Stiefel manifold and by discussing four neural algorithms and a purely algebraic algorithm known from the scientific literature. A comparison of numerical experimental results and computational complexity estimates confirms the effectiveness and efficiency of...
متن کاملComplex-Value Recurrent Neural Networks for Global Optimization of Beamforming in Multi-Symbol MIMO Communication Systems
Multiple antennas at transmitter and receiver can be used to improve communication efficiency by canceling channel noises using the correlated information among the signals transmitted from different antennas. In this paper, a novel approach is proposed for this problem for another interesting case where multiple symbols are used to make the best use of the multiple antenna channel. Such an iss...
متن کامل